The Complexity of Ising Spin Glasses 
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We compute the complexity (logarithm of the number of TAP states) associated with minima and 
index-one saddle points of the TAP free energy. Higher-index saddles have smaller complexities. 
The two leading complexities are equal, consistent with the Morse theorem on the total number of 
turning points, and have the value given in [A. J. Bray and M. A. Moore, J. Phys. C 13, L469 
(1980)]. In the thermodynamic limit, TAP states of all free energies become marginally stable. 



The computation of the complexity of spin glasses has 
recently attracted an avalanche of renewed interest. More 
than two decades ago, two of us (BM) computed the 
average number of solutions, {Ns)j, of the TAP equations 
Pi for the SK Ising spin-glass model. We found {Ng)j ^ 
exp[AfI](T)], where S(T) is the complexity (per spin) 
at temperature T and N is the number of spins. The 
function S(T) vanishes at the spin-glass critical point T^, 
while for T ^ it approaches the value 0.1992... obtained 
from independent (and much simpler) calculations of the 
number of 1-spin- flip-stable states at T = 0, Q . 

Various aspects of this calculation have recently been 
(implicitly or explicitly) criticized S S S H S ll . The 
essence of this criticism is as follows. The TAP equa- 
tions take the form = 0, i = 1, . . . , A, where each Gi 
is a function of the N variables rrii which denote the local 
magnetizations, rrii = {Si), at each site, where the brack- 
ets are thermal averages and Si = ±1 is an Ising spin vari- 
able. The number of solutions of the A^ equations Gi = 
is given by A^^ = ^^^Y{^dmi]\^5{Gi)\<lei dGi/ dmj\, 
where the modulus sign on the determinant ensures that 
each solution is counted with weight unity. In BM (and 
many subsequent calculations) the modulus sign was 
dropped. This is valid if the matrix dGi/drrij is posi- 
tive definite, which requires all TAP solutions to be local 
minima of the TAP free energy. We will argue that, with 
one modification, this idea is essentially correct. This 
seems, at first sight, paradoxical because, if the modulus 
sign is dropped, TAP solution s is weighted by (—1)"% 
where Ug is the index (number of negative eigenvalues) 
of the saddle point, giving A'^s = I)"'- The Morse 

theorem states that this sum is a topological invariant 
equal (in this case) to unity |10| . implying that not all 
TAP states can be minima. We shall resolve this appar- 
ent contradiction by showing that, in the thermodynamic 
limit and within the BM solution, the matrix dGi/druj 
is positive semi-deiinite, with exactly one null eigenvalue. 
For A^ large but finite, we argue that each infinite- A^ so- 
lution splits into a pair consisting of a minimum and an 
index-one saddle point, both types of solution having the 
same (extensive) complexity thereby rescuing the Morse 
theorem. 

Recent work on spin-glass complexity 0, IE IE S 111 
exploits a symmetry of the action, the so-called BRST 
supersymmetry that enters the calculation of A^^. 
If this symmetry is unbroken, the Morse theorem fol- 



lows. However, thr solutions that preserve this symmetry 
have been shown Q to violate the convexity inequality 
(InA^s)j < ln(A^s)j, where the brackets represent dis- 
order averages, leading Crisanti et al. to conjecture that 
there might be no extensive complexity |1] . The BM solu- 
tion breaks the BRST symmetry and satisfies all physical 
requirements, provided the apparent difficulties with the 
Morse theorem can be resolved. It is the purpose of this 
paper to present such a resolution. As a spin-off from our 
calculation, we note that the marginal stability of TAP 
states provides a possible explanation for why they are 
so difficult to find numerically. 

The free energy( multiplied hy (3 — l/ksT) of a TAP 
state is given by 
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where q = {1/N)'^^mf, and the bonds Jij are drawn 
from a Gaussian distribution of mean zero and variance 
1/A^. 

The TAP equations are given by Gi = dF/dmi = 0, 
for alH = 1, . . . , A^, where 



Gi = tanh 



j 



(2) 



Using Gi — 0, the free energy F can be rewritten as a 
sum of single-site terms, F = /i(?ni), where 

/i(m) = -ln2-(l/4)/32(l-g2)_^(-^/2)tanh"^TO 

+ (l/2)ln(l -m^). (3) 

The number of solutions (per unit free-energy range) with 
free energy per spin / = F/N is given by 
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where A is the Hessian matrix. 
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The final term in Eq. (O is 0{1/N) and was omitted 
in BM. It does not contribute to the extensive part of 
the complexity, only to the prefactor of the exponential 
in the relation {Ns{f))j ^ exp[iVI](/)]. It has, however, 
the form of a projector and may play an important role 
in the eigenvalu e sp ectrum of the matrix A, as empha- 
sized by Plefka 131. In particular, it may determine the 
sign of det A. We shall show that, for N oo, the pro- 
jector term splits off a single isolated eigenvalue from the 
continuous spectrum of A. Furthermore, the continuous 
part contains only positive eigenvalues, while the isolated 
eigenvalue is a null eigenvalue outside the continuum. 

Eq. is the common starting point for all calculations 
of Ns{f). Here we focus on the configuration average 
(sometimes called the "annealed average" or "white av- 
erage"), {Ns{f))j, over realizations of the disorder. How- 
ever, it is not straightforward to do this while retaining 
the modulus on the determinant. Dropping the modulus, 
the calculation can be completed and the result takes the 
form Pi 



lln(iV,(/))^ = -Xq-uf-{B + A){l-q) 



where / is a function of the parameters A, q, u, B and A 
and is defined by the integral 
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where P = The left-hand side of © is the complex- 
ity or, more properly, the "annealed complexity" . The 
parameters A,. . . ,A originally entered the calculation as 
integration variables: A and u appear in auxiliary inte- 
grations that relax the delta function constraints on q 
and / respectively, while the other parameters were in- 
troduced via Hubbard-Stratonovich transformations that 
reduce the problem to a single-site problem. Full details 
can be found in Q audi. The resulting five-dimensional 
integral can be evaluated (for N oo) by the method 
of steepest descents, so the five parameters take values 
corresponding to the appropriate saddle point in the five- 
dimensional space. Note that a calculation of the total 
number of solutions, independent of their free energy, re- 
quires setting u — 0. 

It is straightforward to derive the five saddle-point 
equations from Eqs. © and 0. The same five equa- 
tions appear in and 0. Equivalent equations have 
also been derived by De Dominicis et al. The equa- 
tions admit the solution B — Q, and this is the solution 
adopted in all three papers. The differences between the 
various treatments arise in the solution of the remaining 
four equations. 

Consider first the case u = 0, corresponding to a cal- 
culation of the total number of solutions. Cavagna et 



al. 3 (CGPM) note that the BM solution apparently 
violates the Morse theorem, and propose a new BRST- 
symmetric solution that gives vanishing complexity for 
u = 0. As M is decreased, states of lower free energy 
are selected and, within their solution, CGPM find that 
there exists a threshold / below which \n{Ns{f)) j is 
nonzero. Unfortunately, however, an important inequal- 
ity, Xp = l- {(3^/N) X;,(l - mff > 0, is violated in the 
CGPM solution, rendering it unphysical |^. The condi- 
tion a;p > is necessary for the internal consistency of 
the TAP equations [l^[Ti|. This inequality is satisfied 
by the BM solution [g. The BM solution is internally 
consistent, therefore, provided one can demonstrate that 
the matrix A is positive definite, guaranteeing the posi- 
tivity of the determinant and justifying the replacement 
of I det A I by det A in the calculation, and provided one 
can understand the apparent violation of the Morse theo- 
rem that ensues. The remainder of the Letter is devoted 
to these subtle points. 

We first rewrite Eq. (5) in the form Aij = {X^^)ij + 
(2/3^/7V)miTOj, in which the projector term has been sep- 
arated off. The matrix A"-'^ is the susceptibility ma- 
trix, {A~^)ij = dmi/dhj, giving the response to a site- 
dependent external field, and Xij is the 0(1) contribution 
to it. The eigenvalue spectrum of can be obtained 
using cither Pastur's theorem or the 'locator expan- 
sion' of rcf. In the limit N ^ oo, the spectrum con- 
sists of a continuous band of positive eigenvalues for both 
Xp > and Xp < (though, as noted, the TAP equations 
themselves become unphysical for Xp < 0), and the left 
edge of the band reaches zero only for Xp = 0. For the 
(7) BM solution Xp > so all eigenvalues of are pos- 
itive. When the projector term is included, an isolated 
eigenvalue, outside the main band, is produced. U sing 
the eigenvectors of as a basis, it it easy to show [13 
that this eigenvalue has a non-negative value provided 
the inequality 



(6) 



1 > — ^TOiXymj 
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is satisfied, where the final equality defines H. The same 
result can be obtained using the variational trial function 
Vi = Xijmj for the eigenvector of A with smallest 
eigenvalue, i.e. Amm < ^*^y"^j7 Li °^ {l-2f3^H). 
If the inequality ijSJl becomes an equality, the isolated 
eigenvalue Xmin has the value zero, and the variational 
eigenfunction becomes exact. A variant of the inequality 
(O was derived earlier with only the diagonal terms, 
i — i, appearing on the right. The off-diagonal terms 
were missing due to the use of Pastur's theorem outside 
its range of validity p3| . 

The quantity H in Eq. (|SJ) can be computed as follows. 
We introduce an additional factor of unity, expressed as 
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in the integrand of Eq. and obtain H from H = 
{l/N)Y,ij{'>nimj{cj)^(j)j)^)raj where the averages {■■■)^ 
and (. . .)m are over the variables {0^} and {rrii} respec- 
tively. The weight function for the (pi integrals is given 
the integrand in Eq. @, while for the rrii integrals it is 
given by the integrand in Eq. 

After a straightforward but lengthy calculation one 
finds illi 



H 



{q-A,Y+A^[P\{l~q)~A2 



where 



Ai = ((1 — m^)m(tanh ^ m ~ Am)) 
A2 = ((1 - m^)(tanh"^ m - Ato)^) 
^3 - (m2(l-m2)) 



(10) 



(11) 
(12) 

(13) 



and the averages are now over the weight function given 
by the integrand in Eq. Q. Carrying out the required 
integrals numerically (with i? = as usual) one obtains 
a remarkable result: the quantity 20^H is unity for all 
temperatures T <Tc and all values of the free energy per 
spin, /, i.e. the inequality © is satisfied as an equality. 
We feel it should be possible to demonstrate this result 
analytically but thus far have not succeeded. 

This result, that in the thermodynamic limit there is 
always one exactly zero eigenvalue, but no negative eigen- 
value, is the key to resolving all the puzzles surround- 
ing this problem. First, det A vanishes, so the prefactor 
of the exponentially large number of TAP states is, for 
N 00, exactly zero, in accordance with the result of 
Kurchan 10] and its extension to general values of u (the 
variable conjugate to /) 0. However, the exponential it- 
self diverges for TV —> 00, so the product of exponential 
and prefactor is not defined in this limit. To make sense 
of it, one has to keep N large but finite. The result, con- 
firmed by numerical studies, is that the zero eigenvalue 
is shifted, for finite A^, to a small positive or negative 
value, corresponding to a TAP minimum or to a saddle 
of index one respectively. The shift would be expected 
to be of order 17]. No examples of more than 

one negative eigenvalue were found. Furthermore, for a 
given sample the two types of solution typically occur to- 
gether as a closely related pair, in a sense we will clarify 
below. The extrema of the finite- A^ TAP free energy are 
therefore dominated by minima and index-one saddles. 

This picture can be further clarified by constructing a 
Active free-energy function 



Fn=F 
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1) Y.' 
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where is a function of the rrii and q. It is given by 
Eq. but with q regarded as an independent variable, 
unrelated to the rrii, i-e- Fg is a function of the TV -|- 1 
variables mi, . . . , rriN, q, whereas the original TAP free 
energy F depends only on the N variables mi, . . . , m^v 
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FIG. 1: The functions Q{q) (continuous line) and fq{q) 
(dashed line) defined in the text. Physical states occur where 
Q{q) crosses the dotted line Q = q. The date were obtained 
using N = 200 spins at a temperature T — 0.2. 



(with q defined as q = (1/-/V) J^i ^f)- readily veri- 
fies that the stationarity equations for Fq reproduce the 
TAP equations: dFq/dmi = Gi = 0. However, for 
these new equations, the quantity Q = {'^/N)^^mf 
is in general not equal to the parameter q appearing 
in the equations. The additional stationarity equation, 
= dFq/dq = {f3'^/2){Nq ~ ^. mf) forces Q = 9 at sta- 
tionary points in the full (A^ -I- l)-dimensional space. The 
free-energy functions F and Fq have, therefore, the same 
stationary points and the same values at these points. By 
formally eliminating the variables mi, one can obtain the 
function Fq{q) as a function of the single variable q. Its 
first derivative is dFq/dq = [f}'^ /2)[Nq — ^im^), where 
the m, are implicit functions of q through the TAP equa- 
tions. In practice, of course, there will be exponentially 
many TAP solutions, {Ns)j ~ exp[A^Eg], for each fixed 
value of q. Their number can be calculated from the same 
equations, lO and {T)) as before, but with A = 0, since q 
is no longer constrained to equal (1/-/V) mf , and u — 
since / is not fixed. The functions Fq{q) and Q{q), how- 
ever, are self-averaging and therefore well-defined, being 
determined by averages of the appropriate functions of 
m, e.g. Q = {m^) , where the weight function for the av- 
erages is the integrand of Eq. ((T)), with A = = w. 

We have solved these TAP-like equations numerically 
for a range of q, and determined the corresponding val- 
ues of Q and fq — Fq/N . An example is show in Fig- 
ure 1. First a solution of the standard TAP equations 
(i.e. with q = {l/N) m?) was found, and solutions for 
other g-values were generated iteratively from the pre- 
vious value, starting from the TAP solution. The it- 
erative procedure typically fails to coverge when q be- 
comes too small. The physical solutions in Figure 1 



4 



are the two points where the function Q{q) intersects 
the hne Q = q. They correspond to turning points 
of the function fq{q). The solution with the larger q 
always corresponds to a minimum of F, the other so- 
lution to a saddle point of index one. The difference 
vector, Srui, between the solutions typically has a large 
overlap with the eigenvector, e^, of A with the small- 
est eigenvalue: J2i e^f^mj/Ej ef J2ji^i^j)'^V^'^ ~ 0-1 " 1> 
with a typical value around 0.5. This shows that one 
moves from the minimum to the saddle point by mov- 
ing roughly in the direction of the isolated "soft mode" . 
This agrees with our expectation based on the relation 
drrii/dq = P^J^j^ij^j^ which follows from the TAP 
equation. Recall that Vi = J^j becomes, for 

N ~f oo, the null eigenfunction of A, so drrii/dq oc Vi in 
this limit. The minimum and the saddle point will coa- 
lesce as the small eigenvalue tends to zero with increasing 
N , and the two turning points of fq{q) will merge to form 
an inflection point. One can see this formally by differ- 
entiating the relation dfq/dq = {(3"^ /2)[q - (1/iV) X], mf] 
to obtain 

^ = ^(^1-^E™^^^.-.J=0- (15) 

It is important to recall that the isolated eigenvalue 
of order 1 / ^/N does not enter the result for the exten- 
sive part of the complexity, because the projector term 
in Eq. Q responsible for it is 0{1/N) and drops out of 
the complexity at leading order. The upshot is that the 
BM calculation, in which the projector term is neglected, 
counts minima and index-one saddles, both with positive 
sign, since without the projector term the Hessian matrix 
is, for TV — > oo, positive definite. We have shown that in- 
cluding the projector produces one null eigenvalue in the 
thermodynamic limit, i.e. the prefactor in the calculation 
of {Ns)j vanishes as required by exact analysis jS, Jiil- 
For finite N, however, the marginally stable states be- 



come pairs of minima and index-one saddles. 

This suggests a scenario in which the complexities as- 
sociated with minima and index-one saddles are exten- 
sive and equal, and no other solutions are possible in the 
limit N oo except the trivial solution rrii — 0. The 
Morse theorem would then be identically satisfied. Sad- 
dles of index greater than one can only occur if de- 
velops negative eigenvalues for finite N . Then one would 
need to calculate the probability, pk, to have k negative 
eigenvalues. This probability would be expected to be 
exponentially small in N, pk ~ exp(— OfcA'^). Just such a 
computation has been carried out in the (simpler) p-spin 
spherical spin-glass . If the coefficients are large 
enough (larger than the computed complexity of minima 
and index-one saddles), the complexity of index-fc sad- 
dles will be negative and these higher-order saddles will 
not contribute to the Morse sum. 

It has been argued that numerical studies of 
the TAP equation are difficult to reconcile with the BM 
theory since the range of observed free energies is much 
smaller than expected, and the measured value of Xp is 
smaller than predicted. However, the numerical solu- 
tions are obtained from a dynamical algorithm and it is 
known that such algorithms typically generate solutions 
of a given (algorithm-dependent) free energy rather than 
generating them uniformly from the underlying distribu- 
tion [23. Moreover, the free energy of such solutions is 
typically significantly lower than the dominant free en- 
ergy of the underlying distribution, implying that the 
quantity Xp (which decreases as / decreases) will indeed 
be smaller for these dynamically generated states than 
for the dominant states selected by the BM solution. 

We conclude that the BM theory remains a viable can- 
didate theory of spin-glass complexity. 
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